Tomáš Pospíšek's Notizblock
Searching the web brought me to this nice (and broken) article.
In short, if you want to split stdout and pipe it into multiple processes you can do:
echo bar | tee >( grep b > /tmp/foo1 ) >( grep ba > /tmp/foo2 )What does this do?
First, tee does not only allow to write its
stdin to a file and in parallel output it to
stdout as in:
( do stuff ) | tee /tmp/log-file | grep ...but allows to output it to multiple files as in:
( do stuff ) | tee /tmp/log1 /tmp/log2 | grep ...That's the first trick. The second trick is bash's >( )
operator.
Let's take the initial example (and simplify it):
echo bar | tee >( grep ba > /tmp/foo1 )What the >( ) operator does is:
- it creates a named pipe.
On my machine that named pipe looks something like:
/dev/fd/63- it connects that named pipe to the stdin of the subshell that bash created (I'm assuming it is creating a subshell...).
In other words it does something like
cat /dev/fd/63 | grep ba > /tmp/foo1- and then it replaces the whole
>( stuff )thing with the name of the named pipe.
Which gives:
echo bar | tee /dev/fd/63Voilà. Now you can pipe into multiple subshells.
Once again, this blew my mind. I didn't know bash can do this. Bash's man page doesn't waste any needles words on this magnificent feature and covers it in 9 lines. Given bash's 6175 lines of man page there must be unconveivable quantities of unimaginable features still waiting for me to discover ... 8-O!
Update 2021-02-16/17
When your backup job that is using duplicity is whining about deprecated stuff via paramico via the Python cryptography module and you just don't want to hear about it any more:
# cat /etc/cron.d/dubackup
# need bash for the fancy redirection syntax
SHELL=/bin/bash
29 3 * * * root ionice -c3 dubackup 2> >( grep -v DeprecationWarning ) >> /var/log/dubackup.log || echo "backup failed, see /var/log/dubackup.log"Tomáš Pospíšek, 2020-09-29
